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ollege completion rates are low—nearly 40 percent of students 
do not complete a degree within six years of enrolling in college. 
Community college students face the highest hurdles—about 
60 percent are without a degree six years after matriculating.' 
Higher education research has identified strategies that institutions can use 
to improve student success and degree attainment, particularly among stu- 
dents from families with low incomes and others who have been underserved 


by current higher education policies and practices. 


Programs that show significant results must be implemented widely—and with fidelity to 
the proven practice—to meaningfully change student outcomes. Seemingly insignificant 
modifications to effective strategies that colleges implement may diminish the success of 
those initiatives. Research reports often share high-level program descriptions and the 
statistical impact of programs and policies but may not provide enough information on 
their implementation. To support strong implementation of evidence-based strategies, 
MDEC studies and reports on program implementation and provides technical assistance 
to higher education systems and colleges interested in adopting proven practices. 


In 2019, the Minnesota State Colleges and Universities System (Minnesota State), a large 
public higher education system with 30 community colleges and 7 universities that 
collectively enroll more than 200,000 students annually,* partnered with MDRC to 
widely disseminate and help expand evidence-based student success strategies across the 
system. As part of the two-year partnership, leaders from Minnesota State and MDRC 


provided technical assistance with a range of activities, including holding systemwide workshops, 
small-group coaching sessions, facilitating of cross-campus discussions, and creating materials and 
tools for implementation. The series focused on three areas critical to student success: (1) multiple 
measures for course placement, (2) satisfactory academic progress (SAP) communications and poli- 
cies, and (3) comprehensive student support programs. 


For higher education systems interested in supporting implementation of evidence-based strategies 
across their campuses, this brief highlights lessons from the partnership between Minnesota State and 
MDRC. Leaders of individual campuses may also find value in these examples. The brief begins with 
overarching lessons for implementing evidence-based strategies, followed by case studies describing 
specific activities and lessons for the three student success strategies covered during the partnership. 


LESSONS FOR IMPLEMENTING AND SCALING UP PROVEN PRACTICES 


Based on the technical assistance experience and numerous structured conversations with admin- 
istrators from several Minnesota State campuses, this section describes broad lessons in program 
implementation and expansion. 


e Start by selecting strategies that align with existing priorities. 


Deciding which evidence-based strategies to pursue can be difficult, as colleges are often managing 
multiple initiatives. Minnesota State selected three strategies based on the collective interests of cam- 
pus leaders and existing priorities, as shown in Figure 1. Selecting approaches that align with existing 
priorities can increase support for these strategies among faculty and staff and help ensure efforts are 
viewed as a way to achieve existing goals rather than “just another initiative.” Early alignment with 
state, system, and/or institutional priorities can also make it easier to secure ongoing funding and 
increases the likelihood that the selected strategies are sustained beyond the initial pilot, study, or 
grant investment. For these reasons, leaders are encouraged to identify their top student success pri- 
orities and goals and to focus resources on implementing and scaling up complimentary strategies. 


e Create space for collaboration. 


Systemwide change includes multiple players: teams from various campuses within the system, stake- 
holders from various levels and departments, and external technical assistance providers, among 
others. Communication can be difficult. 


To tackle this challenge, MDRC and the Minnesota State leaders set up regular meetings and 
activities with different stakeholder groups to collaborate on technical assistance content and share 
relevant updates and ideas. The partners also used a collaborative document-sharing platform to 
disseminate resources and materials. 
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FIGURE 1 
THREE EVIDENCED-BASED STUDENT SUCCESS STRATEGIES ALIGNED 
WITH SYSTEM PRIORITIES 


Accurately Placing Students Supporting Student Belonging Addressing Equity Gaps Through 


Through Multiple Measures with Improved SAP Comprehensive Student Support 
Assessment Communications and Policies Programs 


SOURCES: *Dan Cullinan, Elisabeth Barnett, Elizabeth Kopko, Andrea Lopez, and Tiffany Morton, Expanding Access to College- 
Level Courses: Early Findings from an Experimental Study of Multiple Measures Assessment and Placement (New York: MDRC, 
2019). 


’Shannon T. Brady, Gregory M. Walton, Omid Futohi, Eric M. Gomez, Geoffrey L. Cohen, and Robert Urstein, “A Scarlet Letter? 
Revising Institutional Messages About Academic Probation Can Mitigate Students’ Feelings of Shame and Stigma,” master’s thesis, 
Stanford University (2019). 


°MDRC (2020). 


College-to-college interaction proved particularly important as the project progressed. Throughout 
the engagement, campus administrators commonly expressed appreciation for time to work with 
staff from their own colleges and to collaborate with their peers at other campuses in the system. 
One important recommendation from the Minnesota project is that leaders planning to imple- 
ment evidence-based strategies should provide both structured and unstructured opportunities for 
intra- and inter-campus interactions and sharing knowledge. 


e Tailor resources to reflect specific institutional contexts. 


Many campus leaders said each of their institutions is unique, which prompted requests for more 
tailored support. This suggests that leaders should consider ways to provide tailored guidance to sub- 
groups, such as rural or urban campuses, and small or large institutions. For example, to implement 
multiple measures to determine students’ academic placements, it is especially important to coordinate 
between college departments responsible for testing, advising, and registration. For Minnesota State, 
this meant coordinating teams from several different departments at larger schools. At smaller col- 
leges, these responsibilities often rested with one individual. Providing specific guidance for various 
institutional contexts can address concerns about the generalizability of evidence-based initiatives and 
support successful implementation across an entire college and university system. To maximize effec- 
tiveness, the guidance offered to each subgroup should remain consistent with available evidence. 
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e Identify early adopters and pilot colleges. 


There is often an initial debate about how best to launch a new strategy, especially when resources are 
limited. Some leaders favor starting something new with a small group, while others advocate starting 
with everyone all at once. With the goal of systemwide change in mind, the workshop series for this 
partnership was open to all colleges. However, throughout the project, Minnesota State and MDRC 
often found tremendous value from featuring colleges that had led or piloted work on the success strat- 
egies and previewing new implementation resources with small workgroups to solicit input and feed- 
back before releasing systemwide changes. The project reinforced the generally accepted best practice 
that pilot campuses or workgroup participants should represent the full spectrum of institutions and 
students that will be served participate in the new strategy so that the effort produces information that 
is useful across the system. 


e Be bold. 


In order to maximize impact, it is important that evidence-based strategies are implemented with fidel- 
ity and thoroughness. “I would encourage others to be bold and to think big from the beginning,” said 
Greg Rathert, the Minnesota State System Director of College Readiness. “There is a growing sentiment 
that we were too conservative in our approach, prioritizing students’ college readiness over access.” His 
call for a robust approach highlights the importance and the potential benefits of rigorous implementa- 
tion from a practitioner’s perspective: 


We reached a point where people were not only comfortable with the framework but wanted 
to go further to be more closely aligned with the research. There was an opportunity to be a 
little bolder and more ambitious... Moving forward, we will take that into account. 


CASE STUDIES: THREE EVIDENCE-BASED STUDENT SUCCESS STRATEGIES 


The remainder of the brief is organized as case studies, describing the research evidence, implemen- 
tation steps to apply the evidence, and specific lessons for each of the three student success strategies: 
multiple measures for course placement, SAP communications and policies, and comprehensive 
student support policies. 


Case Study 1: Using Multiple Measures for Accurate Course Placement 


The Evidence 

Colleges have a long history of using standardized tests to determine students’ readiness for college 
level courses, but research shows these exams are poor predictors of college performance. Up to 
one-third of students put into developmental education through a single placement exam could be 
successful in college-level courses without developmental education.* These students are investing 
time and money on courses that do not earn them credits toward a degree and are associated with 
lower levels of persistence and graduation.® The practice of using multiple measures, or measures 
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that reflect performance over an extended period of time, such as high school GPA, are much more 
accurate at determining the likelihood of a student’s success in college.® Previous publications from 
MDERC and the Center for the Analysis of Postsecondary Readiness, as well as an earlier brief, offer 
more information on using this approach for course placement.” 


Applying Evidence to Practice 

The Minnesota Legislature called for the reform of developmental education and student placement 
in 2017, resulting in a systemwide multiple measures framework.® The framework relies on a decision 
band, which uses high school GPA for students whose placement test scores fall within a predeter- 
mined range. All Minnesota State colleges and universities were charged with implementing the 
framework by the fall 2020 semester, but prior research indicates that implementing multiple meas- 
ures can be challenging. Colleges often struggle to effectively implement the new measures and make 
changes to placement procedures across departments. Other challenges include developing strong 
student processes and communications about multiple measures.° At the time the project launched, 
disparate placement practices were used across the system—adding a layer of complexity to the typ- 
ical challenges of implementing multiple measures. Anticipating these issues, the partners drew on 
implementation lessons from other systems and colleges to support Minnesota State campuses with 
the implementation of the system’s new multiple measures framework. 


Collaborative Steps Toward Implementation. Using a mix of large-group workshops, small-group 
regional meetings, and planning tools, MDRC drew on findings and implementation lessons from 
prior multiple measures evaluations to help participants define their roles in the placement process 
and identify the changes required in order to use the new framework. The partners also helped par- 
ticipants develop reasonable implementation timelines, shared implementation tips, flagged common 
implementation challenges, addressed participant questions, and helped clarify and reform practices 
from various stakeholder perspectives—including the students’ perspective. 


Lessons for Using Multiple Measures in Academic Placement 


In response to challenges presented by the global COVID-19 pandemic, Minnesota State postponed 
the launch of the systemwide multiple measure framework until fall 2021 and developed interim 
placement guidance for campuses within the system. The interim guidance allows campuses to select 
from a range of placement options, including the sole use of high school GPA, which is strongly sup- 
ported by previous research on college placement.’® A subset of Minnesota State campuses may use 
the interim placement guidance beyond fall 2021 to inform future iterations of the systemwide place- 
ment policy. The overall implementation effort yielded these lessons for college systems adopting 
multiple measures." 


e Develop a single multiple measures placement model for the system. 
Minnesota State broadly implemented multiple measures by creating a systemwide decision-band 
placement framework that listed acceptable placement measures and included discipline specific cut 


scores for placement into college-level courses.'? Representatives from colleges and universities across 
the system reviewed and contributed to the framework as it was developed. According to Minnesota 
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State system leaders, a standard, systemwide multiple measures framework offers several benefits: 
First, it advances equity by ensuring students receive similar placements regardless of the institution 
in which they first enroll. Second, it reduces hassles and streamlines placement by providing consist- 
ency for students who transfer between campuses within the system, a common occurrence. Impor- 
tantly, the Minnesota State framework set high-level guidance, but still allowed for some college-level 
decisions. Placement guidelines were ceilings—colleges could utilize new or existing placement 
guidelines that were more generous than those outlined in the systemwide framework. Colleges were 
also charged with outlining guidelines for placement in developmental education courses, as the 
developmental pathways differed by institution. 


e Facilitate development of an automated placement system. 


Prior research on implementing multiple measures placement policies identifies information technol- 
ogy (IT) capability as an important consideration, as automating placement can facilitate implemen- 
tation.'* This consideration is even more critical for implementing multiple measures across a college 
system. System leaders must determine whether to create a systemwide tool to automate placement 
or encourage their campuses to develop their own. Minnesota State developed a systemwide tool to 
automate placement rather than asking individual schools to develop their own. Initially, the system 
experienced delays in creating its tool to automate placement, causing delays in implementation at 
some colleges. However, the system office was ultimately able to create a tool available to all cam- 
puses. Minnesota State’s experience demonstrated that a systemwide automated student placement 
tool can reduce the burden on campuses, increase the likelihood that students across the system have 
similar placement experiences, and speed up implementation. 


e Use peer experiences to build support. 


As the system adopted the multiple measures framework, some faculty and college leaders 
remained hesitant, despite the availability of rigorous research results. Consistent dissemination 
of research findings on the efficacy of multiple measures and opportunities for peer exchange both 
within and outside of the system led to a notable increase in campus faculty and administrators 
accepting multiple measures assessment, especially the incorporation of high school GPA, as a 
reliable tool for course placement. There were two particularly powerful presentations—one from 
a campus within the system that piloted multiple measures and one from the California Commu- 
nity Colleges Chancellor’s office. These sessions highlighted that consistently sharing evidence and 
examples from peers can help develop acceptance and support evidence-based initiatives, even 
among the most apprehensive stakeholders. 


Case Study 2: Supporting Student Belonging with Improved SAP 
Communications and Policies 


The Evidence 
For students from families with low incomes who need financial aid to attend college, meeting 
their colleges’ SAP requirements is critical to persistence, as federal policy stipulates students must 
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meet SAP to remain eligible for aid."* Yet, nationally, about 40 percent of first-year Pell Grant 
recipients at public institutions are at risk of failing to meet these requirements.'* Not meeting 
SAP can diminish students’ sense of belonging in college, causing them to continue to underper- 
form or drop out altogether. 


Research shows a college’s SAP communications, such as letters warning students who do not meet 
SAP requirements, can impact student response. The punitive tone of traditional SAP warning letters 
often leaves students feeling ashamed, discouraged, and unclear about their next steps. Revising let- 
ters to be more empathetic by incorporating psychological principles, including the acknowledgment 
of the numerous personal and structural reasons students struggle academically and testimonials 
from students who regained good academic standing, can improve students’ emotional response to 
letters and, in turn, increase their likelihood of persisting.’® 


Applying Evidence to Practice 

In 2017, as part of the Finish Line: Graduation by Design project, MDRC worked with three Minne- 
sota State colleges to identify institutional and student barriers to meeting SAP. The project found 
that about one-third of students did not meet SAP requirements and received a warning at the end of 
their first semester—only half of these students re-enrolled for their second semester.” Finish Line 
highlighted that variation in the tone of letters notifying students of their SAP status (for example, 
warning or suspension) affected how students responded when they were not meeting SAP standards. 
Some campuses used more positive letters that emphasized steps the students could take to recover. 
At these schools, students had more favorable reactions and were more likely to act on the notices, 

by filing appeals to continue their education, for example. To help disseminate these findings across 
the system, Minnesota State and MDRC focused one strand of the technical assistance series on SAP 
communications and policies. 


Collaborative Steps Toward Implementation. Drawing on national and local research, the partners 
organized activities to help campuses in the system redesign and streamline their academic and 
financial aid warning and suspension letters and identified areas where adjustments to SAP policies 
might help students meet SAP or return to good academic standing if they failed to meet the require- 
ments. The partners first organized two workgroups—one focused on communications and one 
focused on policy—to pilot ideas and help them create resources for the entire system. The work- 
groups comprised representatives from a subset of campuses representative of the system (urban and 
rural, and large and small campuses), system office financial aid leaders, researchers from MDRC, 
and student leaders to ensure resulting materials represented a wide range of perspectives. As shown 
in Box 1, the workgroups created space for larger conversations about the overall SAP process. Addi- 
tionally, MDRC and the system office collaborated with the College Transition Collaborative (CTC), 
a nonprofit organization that bridges research and practice to foster equitable student outcomes in 
higher education, to conduct workshops to share strategies across the system. Through the sessions, 
the partners disseminated findings and implementation lessons from research on SAP, helped cam- 
puses apply evidence-based communication strategies to their warning and suspension notifications, 
raised awareness of the system’s SAP policy, and identified practice adjustments, within the federal 
and system policy, that might improve students’ academic outcomes. 
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BOX 1 


AN EQUITY CONSIDERATION: SINGLE OR DUAL STANDARDS FOR ACADEMIC 
AND FINANCIAL AID REQUIREMENTS? 


The Minnesota State policy workgroup tackled an important equity consideration in SAP policy: Should 
colleges use the same or different standards to assess SAP for academic and financial aid purposes? 
Although system policy requires a single standard, some campus leaders argued that a two-standard 
policy is more student-friendly for those who can afford to pay their tuition and fees out-of-pocket. 
Proponents of a single standard argued that different requirements create the perception that students 
who rely on financial aid have a higher hurdle to overcome than students who can pay out-of-pocket 
and remain enrolled. Though there are benefits to both policies, a single standard may contribute to a 
more equitable environment where all students, regardless of their socioeconomic status, are held to 
the same academic standards. When seeking to improve SAP policy and procedures, it is worthwhile 
to consider how guidelines, as well as the tone and content of SAP communications, might affect 
students in varying circumstances. 


Lessons for Redesigning SAP Communications and Reviewing Policy 


Building on the efforts of the policy workgroup, Minnesota State is currently moving through formal 
procedures to amend its system SAP policy. In particular, the system seeks to exclude developmental 
education credits from SAP calculations, as allowed by federal guidelines. Several campuses across 
the system have also revised their SAP warning and suspension notifications, using the templates or 
guidance from the workshop series. The outbreak of the COVID-19 pandemic illuminated even fur- 
ther the importance of reviewing SAP policies and communications, as students face unprecedented 
challenges to persistence. As such, Minnesota State is continuing to review and revise SAP practices 
to meet students’ evolving needs. The overall effort yielded the following lessons for other systems or 
colleges seeking to improve SAP communications and policies. 


e Provide template materials to facilitate action. 


At Minnesota State, the SAP communications workgroup allowed a representative group of admin- 
istrators, researchers, and students to review research on SAP communications and develop revised 
warning and suspension notice templates, which were shared with all campuses in the system during 
the systemwide workshop. The templates incorporated “psychologically attuned” principles devel- 
oped and tested by CTC, and action-oriented language from the existing SAP letters from one of 

the colleges in the system. This approach of collaborating with a small, diverse workgroup helped 
facilitate wider redesign and implementation of the system’s SAP status notifications. The work- 
group helped ensure the communications templates resonated with students and campus leaders and 
that the materials could be easily used systemwide. For example, the workgroup determined that in 
addition to an academic warning template, creating a combined academic and financial aid warning 
template for campuses that linked the two together would be beneficial. This seemingly small change 
likely helped make implementation of the redesigned communications easier at some campuses. 


Scaling Up Postsecondary Student Success Strategies 8 


e Align written and verbal communication about SAP. 


As campuses across the system began to revise their SAP communications, they addressed the impor- 
tance of rethinking how advisors and other staff verbally communicate with students on SAP warning 
or suspension. It is vital to include advising representatives in the redesign of communications and 
provide them with information, training, and materials to support in-person student interactions. 


Case Study 3: Addressing Equity Gaps in Completion Through 
Comprehensive Student Support Programs 


The Evidence 

Comprehensive student support programs that combine academic, personal, and financial support into 
a single program are one of the most effective strategies for increasing persistence and degree comple- 
tion among students from families with low incomes and traditionally underrepresented students of 
color, according to research."® At their core, these programs use a proactive, case management approach 
to advising or coaching, combined with additional support, which may include financial incentives 

that help students cover daily expenses, such as food or transportation, and enrollment strategies to 
promote timely graduation. Programs for community college students typically last at least three years 
to support them throughout the duration of their academic careers. Program staff members use data to 
monitor student participation and outcomes and drive continuous improvement."® 


Applying Evidence to Practice 

When MDERC began working with Minnesota State, the system focused on reducing large educa- 
tional opportunity gaps in degree completion by race and socioeconomic status. Campus leaders 
chose comprehensive student support programs as a strategy that they wished to explore and adopt 
to address this priority. 


While there was wide interest among campus leaders, adopting and implementing comprehensive 
programs can be complex, and this was the case at Minnesota State. These programs require up-front 
capital to hire and train program directors and coaches and provide financial support to students. 
They also require significant planning and coordination between departments, including academic 
affairs, student services, and business functions. 


Collaborative Steps Toward Implementation. To facilitate the adoption of comprehensive student 
support programs, Minnesota State worked with MDRC to project program costs and return on 
investment and to identify seed funding for campuses. The partners developed and conducted 

a workshop to build knowledge on the efficacy and implementation of comprehensive support 
programs and reviewed important program design and development decisions and considerations 
such as the program model, staffing plan, target student population, launch timeline, alignment 
with strategic initiatives, avenues for sustainability, and evaluation planning. 


Minnesota State elected to join MDRC’s Scaling Up College Completion Efforts for Student Success 
(SUCCESS) initiative, due to its affordability and focus on financial sustainability and scaling up. 
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Four Minnesota State campuses are leading the way, piloting comprehensive student support pro- 
grams for the system, using the SUCCESS program model. At Minnesota State, SUCCESS aligns 
with the system’s Equity 2030 goal to reduce stark opportunity gaps by race and socioeconomic sta- 
tus.?° Programs are being designed to serve students from underrepresented populations, including 
students from families with low incomes, students of color, and first-generation students. The system 
office appointed its Director of Student Development to lead SUCCESS and coordinate between cam- 
puses, and MDRC is currently providing campuses with technical assistance to design and imple- 
ment their programs according to the SUCCESS model. 


Lessons for Launching Comprehensive Support Programs 

Minnesota State is in the early phases of launching SUCCESS. The four pilot campuses will begin 
serving students in fall 2021. Program design with the Minnesota State pilot campuses revealed sev- 
eral considerations for college or university systems looking to implement and scale up comprehen- 
sive student support programs. 


e Identify seed funding to launch pilots. 


One of the biggest barriers to the implementation of comprehensive support programs is cost. While 
the programs have been shown to be cost-effective, colleges must still identify up-front funding for 
the program.?' Minnesota State is funding its SUCCESS pilots through a dedicated pool of funding 
to support cross-campus collaboration and enterprise-wide initiatives created by a 1 percent holdback 
of the system’s state allocation. This funding largely covers program costs in the first year of the pro- 
gram, but colleges will increasingly contribute their own institutional funds. This type of innovative 
funding structure could be a model for other college systems interested in piloting comprehensive 
supports. Identifying funding is a critical role that a system office might play to implement compre- 
hensive supports across the system. Box 2 suggests additional roles that system leaders might play. 


BOX 2 


HOW SYSTEM LEADERS CAN HELP IMPLEMENT COMPREHENSIVE STUDENT 
SUPPORT PROGRAMS 


e Identify partnership and technical assistance opportunities. 

e With input from campus leaders, select a program model that allows for local adaptations. 
e Secure seed funding for campuses to pilot the program. 

e Facilitate sharing of resources and implementation supports. 


e Develop a plan to expand the program to other campuses. 
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e Determine which program features will be systemwide versus institution-specific. 


Higher education systems seeking to adopt and scale up comprehensive student support programs 
across a system must tackle an important question: which program features will be standard across 
all campuses, and which will be specific to individual institutions. Standard program features help 
to create a common experience for students at different campuses, while institution-specific features 
could reflect campus autonomy. For example, one question that arose while launching SUCCESS 
with Minnesota State was whether the pilot campuses would adopt one name and brand for the 
program, or each select its own. Individual program names could reflect local campus identity; how- 
ever, a single program name might create a more visible brand that students could recognize across 
campuses and potentially allow students in the program who transfer from a community college to a 
university in the system to continue receiving support. Ultimately, Minnesota State decided to select 
one program name and brand. However, other decisions, such as how to operationalize various com- 
ponents of the SUCCESS program model, were left to the individual campuses. 


CONCLUSION 


The partnership between MDRC and Minnesota State expanded the adoption of the highlighted stu- 
dent success strategies by more campuses across the Minnesota State system. College collaboration, 
along with coordination and financial support from the system office helped to accelerate change. 
The work is ongoing, as Minnesota State continues to work together as a system and continuously 
improve multiple measures for course placement, SAP communications and policies, and compre- 
hensive student support programs. 


Colleges and systems looking to adopt or scale up multiple measures for course placement, rede- 
sign SAP communications and policy, or launch comprehensive student support programs can use 
these lessons as a starting point. While the activities detailed in this brief describe the work com- 
pleted for this project in Minnesota, MDRC’s work with colleges covers a broad span of topics and 
evidence-based interventions. MDRC collaborates to develop meaningful programs that support 
student success; facilitate program implementation; evaluate their efficacy through randomized 
controlled trials; and identify opportunities to disseminate findings through journals, briefs, confer- 
ences, and other channels. To learn more about MDRC’s work with postsecondary institutions, see 
https://www.mdrc.org/issue/higher-education. 
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